Skip to content

[8.2](backport #31798) [libbeat] Improve syslog parser/processor error handling#32117

Merged
taylor-swanson merged 1 commit into8.2from
mergify/bp/8.2/pr-31798
Jul 5, 2022
Merged

[8.2](backport #31798) [libbeat] Improve syslog parser/processor error handling#32117
taylor-swanson merged 1 commit into8.2from
mergify/bp/8.2/pr-31798

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Jun 27, 2022 •
edited by zube bot
Loading

This is an automatic backport of pull request #31798 done by Mergify.
Cherry-pick of cabc8ba has failed:

On branch mergify/bp/8.2/pr-31798
Your branch is up to date with 'origin/8.2'.

You are currently cherry-picking commit cabc8badb9.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.next.asciidoc
	modified:   NOTICE.txt
	modified:   go.mod
	modified:   go.sum
	modified:   libbeat/processors/syslog/docs/syslog.asciidoc
	modified:   libbeat/processors/syslog/syslog.go
	modified:   libbeat/reader/syslog/docs/syslog.asciidoc
	modified:   libbeat/reader/syslog/parser/common.rl
	modified:   libbeat/reader/syslog/parser/parser_rfc3164.rl
	modified:   libbeat/reader/syslog/parser/parser_rfc5424.rl
	modified:   libbeat/reader/syslog/parser/rfc3164.rl
	modified:   libbeat/reader/syslog/parser/rfc5424.rl
	modified:   libbeat/reader/syslog/rfc3164_gen.go
	modified:   libbeat/reader/syslog/rfc3164_test.go
	modified:   libbeat/reader/syslog/rfc5424_gen.go
	modified:   libbeat/reader/syslog/rfc5424_test.go
	modified:   libbeat/reader/syslog/util.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   libbeat/processors/syslog/syslog_test.go
	both modified:   libbeat/reader/syslog/message.go
	both modified:   libbeat/reader/syslog/message_test.go
	both modified:   libbeat/reader/syslog/syslog.go
	both modified:   libbeat/reader/syslog/syslog_test.go
	both modified:   libbeat/reader/syslog/util_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

@mergify mergify bot requested a review from a team as a code owner June 27, 2022 16:28
@mergify mergify bot requested review from belimawr and cmacknz and removed request for a team June 27, 2022 16:28
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Jun 27, 2022
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 27, 2022
@botelastic
Copy link

botelastic bot commented Jun 27, 2022

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 27, 2022 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-05T16:18:23.595+0000

  • Duration: 98 min 1 sec

Test stats 🧪

Test Results
Failed 0
Passed 22666
Skipped 1941
Total 24607

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify
Copy link
Contributor Author

mergify bot commented Jul 4, 2022

This pull request has not been merged yet. Could you please review and merge it @taylor-swanson? 🙏

@taylor-swanson taylor-swanson force-pushed the mergify/bp/8.2/pr-31798 branch from 1f34b95 to f197c15 Compare July 5, 2022 15:22
@taylor-swanson
[libbeat] Improve syslog parser/processor error handling (#31798)
c7e898f 
- Shifted most of the validation logic to the host language instead of
the Ragel parser. This allows for the message to be fully parsed even if
there are validation errors with one or more fields.
- Errors are accumulated as they occur, any fields that have already been
parsed and validated will be returned, alongside error.message.
- Priority field for RFC 3164 messages are now optional, as some
syslog providers will omit this part of the header (especially when writing to files).
- Structured data fields for RFC 5424 messages are processed in a
second pass. If the structured data format doesn't fit the RFC, it will be
prepended to the message, separated by a space.
- Structured data now uses map[string]interface{}
- Update testify to v1.7.1 dependency for assert.ErrorContains support

(cherry picked from commit cabc8ba)
@taylor-swanson taylor-swanson force-pushed the mergify/bp/8.2/pr-31798 branch from f197c15 to c7e898f Compare July 5, 2022 16:17
@taylor-swanson taylor-swanson merged commit 5ae31be into 8.2 Jul 5, 2022
@taylor-swanson taylor-swanson deleted the mergify/bp/8.2/pr-31798 branch July 5, 2022 18:46
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@mergify @taylor-swanson
[libbeat] Improve syslog parser/processor error handling (elastic#31798…
35796e8 
…) (elastic#32117)

- Shifted most of the validation logic to the host language instead of
the Ragel parser. This allows for the message to be fully parsed even if
there are validation errors with one or more fields.
- Errors are accumulated as they occur, any fields that have already been
parsed and validated will be returned, alongside error.message.
- Priority field for RFC 3164 messages are now optional, as some
syslog providers will omit this part of the header (especially when writing to files).
- Structured data fields for RFC 5424 messages are processed in a
second pass. If the structured data format doesn't fit the RFC, it will be
prepended to the message, separated by a space.
- Structured data now uses map[string]interface{}
- Update testify to v1.7.1 dependency for assert.ErrorContains support

(cherry picked from commit 7ba9a7b)

Co-authored-by: Taylor Swanson <90622908+taylor-swanson@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@belimawr belimawr Awaiting requested review from belimawr belimawr is a code owner automatically assigned from elastic/elastic-agent-data-plane

@cmacknz cmacknz Awaiting requested review from cmacknz cmacknz is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@taylor-swanson taylor-swanson

Labels

backport conflicts There is a conflict in the backported pull request needs_team Indicates that the issue/PR needs a Team:* label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@elasticmachine @taylor-swanson